Protocol layering is fairly common in computer communications over data networks. For example, the well-known protocols used for data communications over the Internet, TCP/IP, consist of various protocol layers stacked on top of one another. Each protocol layer offers its clients differing functionalities and communication performance. Such layering is typically used, among other things, to insulate software layers which utilize protocols for reliable communication from the physical hardware details that must be considered for actually sending digital data over the network.
Another example of protocol layering in high-performance parallel systems is utilized in IBM's RS/6000® SP™ systems. (IBM and RS/6000 are registered trademarks of International Business Machines. SP is a common law trademark of International Business Machines.) In this example, two messaging layer protocols are provided in IBM® RS/6000® SP™ systems. These messaging layer protocols are a Message Passing Interface (MPI) and a low-level application program interface (LAPI). (Low-level application program interface and LAPI are trademarks of International Business Machines Corporation.) Each of these message layer protocols has a separate application program interface (API) that can be accessed from a user program. Each of these protocol layers sits on top of a Hardware Abstraction Layer (HAL) interface that interacts with the network interface and device driver to send the data over the physical network. The HAL interface provides both MPI and LAPI with a standard interface library that hides implementation details of the network interface and device driver from both MPI and LAPI. MPI provides users with a two-sided communication model using an industry-standard API. LAPI provides users with an active message interface that is inherently one-sided and asynchronous; it provides users with enhanced communication performance with a custom IBM API.
Layering an industry-standard message layer protocol, such as MPI, over LAPI can provide users with many of LAPI's performance advantages while coding to an industry standard. An interface to LAPI which facilitates layering MPI over LAPI is provided by enabling LAPI to call routines provided by a client MPI program to process the data of messages received by LAPI which are addressed to the client MPI program. However, enhancements to the interface of an active message layer protocol which facilitate such layering can improve data communications efficiency where an active message layer protocol such as LAPI serves as a substrate messaging layer protocol for other message-layer protocol implementations, such as MPI protocol implementations.